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(57) ABSTRACT 

Embedding a watermark in an image by changing selected 
DCT coefficients in the blocks and macro blocks of coeffi- 
cients which represent the image. The changes in the blocks 
that comprise each macro block are done in a coordinated 
manner so that the phase of the watermark signal is pre- 
served across the block boundaries. By preserving the phase 
across block boundaries, a detectable grid is formed which 
can be used as an orientation and scaling grid. Furthermore, 
by preserving the phase across block boundaries the visual 
artifacts introduced by the watermark are minimized. The bit 
rate of the image signal is preserved by maintaining a count 
(referred to as the cumulative change count) that represents 
the amount that the bit rate has been increased by changes 
in coefficients less the amount that the bit rate has been 
decreased by changes in the coefficients. If at any time the 
cumulative change count exceeds a pre-established limit, 
coefficient changes that decrease the cumulative change 
count continue; however, coefficient changes that increase 
the cumulative change count arc suspended. The suspension 
of coefficient changes that increase the cumulative change 
count continues until the cumulative change count falls 
below the pre-established limit. 

2 Claims, 4 Drawing Sheets 
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METHOD AND APPARATUS FOR 
WATERMARKING VIDEO IMAGES 

RELATED APPLICATIONS 
This application is: 

1) a continuation in part of Patent application No. 60/056, 
968, which was, filed on Aug. 8. 1 997 and which is now 
pending, 

2) a continuation in part of application Ser. No. 08/746, 
613 filed Nov. 12, 1996 which is now U.S. Pat. No. 
6,122,403 and 

3) a continuation in part of application Ser. No. 08/649, 
419 which was filed May 16, 1998 which is now U.S. 
Pat. No. 5,862,260. 

FIELD OF THE INVENTION 

This invention relates to stenography, to copy protection 
and to applying and detecting digital watermarks in video 
images. 

BACKGROUND OF THE INVENTION 

The advent of technology for storing images on digital 
media has increased the need for a method to protect against 
piracy. Images stored on prior forms of media (e.g. VHS, 
Beta, audiotapes, etc) are inherently degraded when copied. 
Images stored on digitally encoded media can be copied 
with no degradation; therefore, perfect copies of copies of 
copies, etc. can be made. 

The introduction o£ Digital Versatile Discs (DVD) con- 
taining movies has created increased incentives for both 
casual and professional unauthorized copying. At the movie 
industry's urging, technology has been put in place to 
protect against simple duplication of DVD disks using 
equipment avaUable to unsophisticated consumers. This is 
similar to the protection that exists which prevents one from 
duplicating a VCR tape by connecting together two com- 
mercially available VCRs. 

While such protection mechanisms protect against some 
types of copying, a personal computer connected to a DVD 
device present a much more complicated problem. Open 
architecture devices such as personal computers reproduce 
the signals in the "clear" and such devices have many entry 
points, which can be used to duplicate material once it is in 
the "clear". The present invention uses digital watermarks to 
address the above described problem. The present invention 
also has other applications. 

It is known that to facilitate the detection of digital 
watermarks one can insert a watermark signal that forms a 
grid. The grid can be used to determine orientation and scale. 
With the present invention the data signal and the grid signal 
are integrated into a single watermark signal in such a 
manner that the visual artifacts introduced by the watermark 
are minimized. 

In applications such as DVD, an important factor that 
needs be considered is the bit rate of the bit stream. There are 
disadvantages if introduction of a watermark into a bit 
stream changes the bit rate. For example if images are going 
to be recorded on a medium such as a DVD disc, increasing 
the number of bits in the bit stream will decrease the number 
of images that can be recorded on a single disk. It is known 
that, in general, adding a watermark to a stream of images 
will increase the number of bits in the bit stream. The present 
invention provides a method and apparatus, which preserves 
the bit rate even though watermarks are introduced into the 
images. 
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SUMMARY OF THE INVENTION 

The well-known JPEG and MPEG data compression 
techniques transform images utilizing a direct cosine trans- 
form (DCT) which produces a matrix of DCT coefficients. 
These coefficients are arranged into blocks (e.g. into 8 by 8 
blocks of coefficients). The blocks of DCT coefficients are in 
turn arranged into macro blocks (e.g. into 16 by 16 arrays 
containing four 8 by 8 blocks). With the present invention 

J0 selected DCT coefficients in each block are slightly 
increased or slightly decreased in response to a watermark 
signal. The changes in the blocks that comprise each macro 
block are done in a coordinated manner so that the phase of 
the watermark signal is preserved across the block bound- 

15 aries. By preserving the phase across block boundaries, a 
detectable grid is formed which can be used as an orientation 
and scaling grid. 

The present invention also maintains the bit rate of the 
image signal. The bit rate of the signal is preserved by 

20 maintaining a count (referred to as the cumulative change 
count) that represents the amount that the bit rate has been 
increased by changes in coefficients less the amount that the 
bit rate has been decreased by changes in the coefficients. If 
at any time the cumulative change count exceeds a pre- 

25 established limit, coefficient changes that decrease the 
cumulative change count continue; however, coefficient 
changes that increase the cumulative change count are 
suspended. The suspension of coefficient changes that 
increase the cumulative change count continues until the 

3Q cumulative change count falls below the pre-established 
limit. The above described process can be described as 
selectively changing the intensity of a watermark signal in a 
bit stream so as to prevent the entropy of the combined 
signal from exceeding a pre-established limit. 

35 BRIEF DESCRIPTION OF THE FIGURES 

FIG. 1 is a diagram illustrating how the pixels in an image 
are arranged into blocks and how the resulting DCT coef- 
ficients are numbered. 
40 FIG. 2 is a diagram, which shows how the blocks of DCT 
coefficients are arranged into macro blocks. 

FIG. 3 is a program flow diagram showing how the 
coefficients in each block of a macro block are treated so as 
to preserve the phase of watermark signal in each macro 
45 block. 

FIG. 4 is a program flow diagram showing how the bit rate 
in the data stream is maintained constant. 

DESCRIPTION OF PREFERRED 
50 EMBODIMENTS 

The well know MPEG (Motion Picture Expert Group) and 
JPEG (Joint Photographic Expert Group) image compres- 
sion techniques use a DCT (Discreet Cosine Transform) to 

55 generate a matrix of coefficients. The preferred embodiment 
of the invention shown herein slightly modifies the DCT 
coefficients (either slightly increases or slightly decreases 
the value of the coefficients) so as to embed .a. digital 
watermark in the image. Such a digital watermark can later 

60 be detected by conventional cross correlation techniques. 
As illustrated in FIG. 1, the MPEG and JPEG techniques 
divide an image into 8 by 8 blocks of pixels. Each block of 
pixels is then used to generate an eight by eight block of 
DCT coefficients. The 8 by 8 blocks of coefficients are 

65 divided into "macro blocks", each of which consist of four 
of the original blocks. This is illustrated in FIG. 2. The rows 
and columns of DCT coefficients in each block are num- 
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bered from top to bottom and left to right as illustrated in detected using conventional watermark detection tech- 

FIG. 1. The first row and the first column are designated as niques. Thus, the watermark data itself is used to form the 

the "0" row and "0" column. grid that can be used to determine scale and rotation. 

Certain of the DCT coefficients in each block are selected FIG. 3 is an overall program flow diagram of the above 

as the coefficients that will carry a selected bit of the digital S descri5ed aspect of lhe preferred embodiment The system 

watermark signal In the preferred embodiment the three fe a stream of data ^ tm { mo(± m 

coefficients circled in FIG. 1 are used to carry the first or "0" . , , . , . * 4 

bit of the watermark data signal. These three coefficients are ^conventional device or program module that generates 

modified, that is, either slightly increased or slightly DCT for the ima £ es «> the data stream. These 

decreased depending upon the value of the "0" bit of the io coefficients are sent to a decision unit 302 which separates 

watermark data. In a similar manner other coefficients are the data int0 macro blocks and sends il t0 units 311, 312, 313 

slightly changed in order to carry the other bits of the and 314 depending upon whether the data represents a block 

watermark signal. 1, 2, 3 or 4 in a macro block. Units 311-313 modify the DCT 

:0he aspect of the present invention is directed to insuring coefficients in order to imbed a watermark signal according 

that the sinusoids generated by the changes made to the DCT 15 to the following rules. 

coefficients are continuous, that is, in-phase across the four Unit 311: modify the coefficients in a conventional man- 
blocks that constitute each macro block. First, if the sinu- _ > ■ „ . . 

. , . , , . , ner to imbed watermark, 
soids that carry the watermark are continuous across each 

macro block, there will be less edge effects and the water- Unit 312: invcrt thc diction of the change if the coef- 

mark will be less visually noticeable. Second, the sinusoids 20 ficient is in an odd row. 

which are continuous over the four blocks of each macro Unit 313: invert the direction of the change if the coef- 

block create a low level orientation or grid signal. This low ficient is in an odd column 

level grid signal can be detected to determine the orientation „ . t . 4 ,. f iL , . , iL - 

j i r .u . it-u • i , t , , Unit 314: invert the direction of the change id the coef- 

and scale of the watermark. The grid signal can then be _ . . . %t . . . , , , 

detected using the cross correlation techniques. Cross cor- 25 ficient LS in an odd row or lf 11 15 in and odd column > but 

relation detection techniques are for example shown in do not inverl the d *"**ion ^ the change if the coeffi- 

copending patent application 08/649,149 filed May 16, 1996 cient is in both and odd row and in an odd column. 

and in issued U.S. Pat. Nos. 5,748,763 and 5,748,782. The output of units 322 to 314 is combined by unit 320 

If certain DCT coefficients in adjacent blocks are modified back into a single data stream. It is noted that each of the 

in the same direction, the resulting sinusoids will not be 30 units shown in FIG. 3 could be separate units, which are 

continuous across block boundaries. With the present inven- either programmed, or hardwired to perform the specified 

tion the changes made to the coefficients of each of the four functions. Alternatively all the function could be performed 

blocks in a macro block are coordinated so that the resulting m a s j ng i e programmed computer on a time-shared basis. 

sinusoids will be continuous across block boundaries within 3s The unit which generales DCT coc fficients is conventional 

each macro block. The changes are coordinated using the and guch units are knQw and nQt of ^ { ^ 

rules explained below. 

The blocks in each macro block are numbered as shown ~£ . , ... . , 

• i-i^ i ni 1 .j j • . t ™ The -previous discussion describes how a watermark can > 

in FIG. 2. Block one is considered the base block. The , , . , , . . „™ , . , . 

coefficients in this block are changed in a conventional way '-J* Produced in the DCT domain. It is noted, that the 

by the associated bits of the watermark signal. Note, the lability of the overall watermarking can be increased by 

following paragraphs relate to how the coefficients which usin g watermarks. One watermark can be added by 

are circled in FIG. 1 are changed in response to the "0" bit /modification of the pixels in the original image in the ; 

of the watermark. It should be understood that other coef- 1 manner as described in U.S. Pat, Nos. 5,748,763 or 5,748, 

ficients must be similarly changed to carry the other bits in 45 783-and then a second watermark can be added by modifi- ~> 

the watermark data. cation of the coefficients in the DCT domain as described^ 

In block 1, the coefficients that are circled in FIG. 1 are herein. ■■-■-> 

slightly increased or slightly decreased in response to the Another problem ad dressed by the present invention is the 

"0" bit of the watermark data. In blocks 2, 3 and 4, the _ ■ , • „ u;# rnta • „ nf 

. . . -~. , -r™* . , ■ need to maintain a constant bit rate in a stream or bits 

circled coefficients shown in FIG. 1 are changed in response c n r • i( _ L A . 

„ t . ... c.. t , . r ii • au representing a senes or images even though watermarks are 

to the zero bit of the watermark according to the followine f , , f r . , . , 

^ 6 B added to the images. It is noted that MPEG and JPEG 

Block 2: invert the direction of the change if the coeffi- ™ variable len S* codes * «P«f * data, hence, 

cient is in an odd row adding watermarks generally increases the bit rate of a data 

Tji i •% > ,i j * * . r 4 t . . c ( . az stream. Typical a watermark has no correlation with the 

Block 3: invert the direction of the change if the coeffi- 55. • L - i . . JJJL LJ 

cient is in an odd column iraa S e int0 wmch lhe walermark 15 embedded, thus embed- 

Block 4: invert the direction of the change if the coeffi. d ^ «> image in a watermark produces an image which has 

cient is in an odd row or if it is in and odd column, but * m S her entro Py than lhe on S inal ima S?< ^ blt rate of a 

do not invert the direction of the change if the coeffi- data stream transmitting an image correlates directly to the 

cient is in both and odd row and in an odd column. 60 entro Py of the ima g e - 

If the above rules are followed the sinusoids generated by Typically the number of codes used to code an image, that 

the change in the DCT coefficients will be continuous across ^ the number of entries in the Huffman table of a coded 

the boundaries in the four blocks that form each macro image, is relatively large (e.g. 500). However, the changes 

block. These sinusoids will be able to be detected using that occur when a watermark is introduced into an image can 

conventional cross correlation techniques and they can be 65 be illustrated with following simple example. Consider a 

used as a grid to determine the scale and rotation of the data stream that has only four symbols, si, s2, s3 and s4, 

image. The data bits in the watermark will also be able to be which are encoded as follows: 
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Symbol code 


si 


0 


S2 


01 


S3 


110 


S4 


111 



Then consider a data stream as follows: 

Bit stream: 0011010111010 

Decoded stream 0/0/110/10/111/0/10 

Decoded message: si, s2, s3, s2, s4, si, s2 

When a watermark is added to an image the bits in the 
image are slightly changed. In the above simplistic illustra- 
tive example, in some situation the symbol s2 would be 
changed to the symbol s3 and hence the number of bits in a 
bit stream which transmits the image would be increased. In 
fact there are mathematical principles (not explained herein) 
which show that when a normally distributed watermark 
(that is, a watermark with a Gaussian distribution) is added 
to an image, and the image is transmitted using variable 
length Huffman codes, the length of the bit stream will of 
necessity be increased. 

The present invention provides a technique for insuring 
that when a watermark is added to a data stream, the bit rate 
will be maintained constant. It is noted that the present 
invention does not violate the above-described mathematical 
principle, because with the present invention, some of the 
redundancy normally used to watermark images is in certain 
circumstances decreased. That is in certain circumstances 
the intensity of the watermark is decreased. 

With the present invention, the watermark is modified in 
response to characteristics of the image. Thus, to some 
extent the watermark is correlated to the image into which 
the watermark in embedded. In this way a watermark can be 
embedded into an image and the entropy of the combined 
image and watermark will be substantially equal to the 
entropy of the watermark alone. 

With the present invention, the system maintains a cumu- 
lative count of the amount that the coefficients have been 
changed to any point in time. That is, the amount of positive 
changes less the amount of negative changes made since the 
beginning of the bit stream is tracked. This amount is herein 
referred to as the cumulative change count. If at any time, 
the cumulative change count exceeds a pre-established posi- 
tive limit, no further positive changes are made. 

Normally it is only necessary to insure that changes do not 
increase the bit rate unduly; however, in some instances it 
may also be desirable to insure that changes do not unduly 
decrease the bit rate. If this is desired, the same technique as 
described above can be used to insure that the cumulative 
change amount does not exceed a pre established negative 
limit. That is, if the cumulative change amount exceeds a 
pre-established negative value, positive changes continue in 
a normal manner, but no further negative changes are made. 

The magnitude of the pre-established maximum (and in 
both a positive and negative direction) are established at the 
values which constitutes the change in bit rate which can be 
tolerated in a particular system. 

FIG. 4 is a program flow diagram showing how the data 
rate is maintained constant not withstanding the fact that 
watermarks are added to the images in the data stream. 
Block 403 A shows that a limit on the amount of positive 
changes that can be made to DCT coefficients is established 
and stored. Blocks C shows that the cumulative change 
amount is stored. The cumulative change amount is the 
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amount of positive changes less the amount of negative 
changes that have been made to coefficients since the start of 
the data stream. 
The DCT coefficients are calculated in the normal manner 

S as indicated by block 401. Likewise the change in each the 
DCT coefficients needed to embed the watermark is also 
calculated in the normal manner as shown by block 405. 
Block 405 shows that a check is made to determine if the 
needed change in a particular DCT coefficient is positive or 

10 negative. Block 409 indicates that if the change is positive 
a check is made to determine if the maximum allowable 
cumulative change amount stored in block 403A will be 
exceeded if the change is made. 

Blocks 415, and 417 indicate that the coefficients will 

15 only be changed, if the change does not cause the cumulative 
change amount to exceed the limit in 403A. Finally as 
indicated by blocks 425 and 427, the cumulative change 
amount in register 403C is incremented or decremented if a 
change to the coefficients is in fact made. Block 431 

20 indicates that the coefficients are sent to the output of this 
process and they are then transmitted and processed in a 
normal manner. 

It is noted that the present invention relates to embedding 
a watermark in an image. Various know techniques can be 

25 used to detect watermarks embedded in images utilizing the 
present invention. For example techniques such as those 
described in U.S. Pat. Nos. 5,748,763, and 5,748,783 or in 
the "Communications of the ACM" July 1998/vol. 41, No.7. 
or in pending applications Ser. No. 08/746,613 filed Nov. 12, 

30 1996 and Ser. No. 08,649,419 which was filed May 16, 1998 
(all of which are hereby incorporated herein by reference) 
could be used. 

While the process has been described above as one where 
a change is either made or not made, it should be understood 

35 that alternatively, the amount of the change could be 
decreased if the limit in the cumulative change value is being 
approached. It is also noted that the system shown in FIG. 
4 prevents the cumulative change value from exceeding a 
pre established positive limit. Since adding a watermark to 

40 an image generally increases the entropy of the image and 
since Huffman code tables are normally constructed such 
that an increase in entropy result in increased bit rate, the use 
of only a positive limit is normally appropriate. However, in 
some situations, it may be appropriate to tract if the cumu- 

45 lative change amount exceeds a limit in both the positive and 
negative directions. Such a check could be added to FIG. 4 
prior to block 427. 

It is recognized that by implementing the present 
invention, the strength of the watermark is in some cases 

so reduced. However, the reduction is not sufficient to prevent 
detection of the watermark. The changes made with the 
above invention merely lower the intensity of the watermark 
in a selective manner, thus in some instances more process- 
ing may be required to detect the watermark. 

55 In many systems, each Huffman code covers several 
symbols. In such systems the calculation indicated by block 
405 is not the change in a single symbol that results from 
adding a watermark to the image. In such systems the 
calculation indicated by block 405 is a calculation of the 

60 change that results in the bit string of whatever combination 
of symbols used in the Huffman code to represent a symbol. 
In some cases the calculation might have to be done over 
several combinations of symbols. 

It is also noted that various aspects of the present inven- 

65 tion are shown herein in a single preferred embodiment. 
Other alternative embodiments could use one but not all 
aspects of the present invention. For example the part of the 
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present invention that relates to maintaining bit rate could be 
used in embodiments which do not use macro blocks to 
establish an orientation grid. Likewise the aspect of the 
present invention which relates to the use of macro blocks 
could be used without the part of the invention that relates 5 
to maintaining a constant bit rate. Finally, while the inven- 
tion has been shown in an embodiment that inserts a 
watermark in the DCT domain, the invention could be used 
in applications where watermarks are inserted in other 
domains. 

While the invention has been shown and described with 
respect to preferred embodiments of the invention, various 
changes in form and detail could be made without departing 
from the spirit and scope of the invention. The applicants 
invention is limited only by the appended claims. 
We claim: 15 
1. A method for adding a multibit watermark to an image 
comprising, 

generating DCT (Discrete Cosine Transform) coefficients 
representing said image, said coefficients being 
arranged in blocks, said blocks being arranged into 20 
macro blocks, 

each of said macro blocks containing a block one, a block 
two, a block three and a block four, and the coefficients 
in each block being arranged in rows and columns, 
alternate of said rows and columns being designated 25 
even and odd rows and columns, adjusting said coef- 
ficients as follows: 

the coefficient in block one being changed directly in 
response to the bits of said watermark, the bits in blocks 
two, three and four being changed in accordance with 30 
the following rules: 

Block two: invert the direction of the change if the 
coefficient is in an odd row, 

Block three: invert the direction of the change if the 
coefficient is in an odd row column 35 

Block four: invert the direction of the change if the 
coefficient is in an odd row or if it is in and odd 
column, but do no invert the direction of the change 
if the coefficient is in both and odd row and in an odd 
column, 
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whereby the sinusoids generated by said adjustments are 
in phase across block boundaries in each macro block. 
2. A system for adding a multibit watermark to an image 
comprising, 

means for generating DCT (Discreet Cosine Transform) 
coefficients representing said image, said coefficients 
being arranged in blocks, said blocks being arranged 
into macro blocks, 
each of said macro blocks contains four blocks, block one, 
block two, block three and block four, the coefficients 
in said blocks being arranged in rows and columns, 
alternate of said rows and columns being designated 
even and odd rows and columns, 
means for adjusting said coefficients in accordance with 
the bits of said watermark to embed said watermark in 
said image, 

the coefficient in block one being changed directly in 
response to the bits of said watermark, the bits in blocks 
two, three and four being changed in accordance with 
the following rules: 

Block two: invert the direction of the change if the 

coefficient is in an odd row, 
Block three: invert the direction of the change if the 

coefficient is in an odd column 
Block four: invert the direction of the change if the 
coefficient is in an odd row or if it is in and odd 
column, but do not invert the direction of the change 
if the coefficient is in both and odd row and in an odd 
column 

whereby the sinusoids generated by said adjustments are 
in phase across block boundaries in each macro block, 
and 

whereby the same changes which represent said water- 
mark form a grid which can be used to detect the scale 
and rotation of said watermark. 

* * * * * 
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